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METHOD AND APPARATUS FOR LOCATING OR REDISCOVERING PARTIALLY 

OBSCURED OBJECTS IN IMAGES 

BACKGROUND AND SUMMARY OF THE INVENTION 

[0001] This application claims the priority of German patent 
document 199 02 681.5, filed 23 January 1999 (23.01.99) and PCT 
International Application No. 99/DE99/04064 , filed 22 December 
1999, the disclosure of which is expressly incorporated by 
reference herein. 

[0002] This invention relates to a method and apparatus for 

locating or rediscovering partially obscured objects in images, 
by correlation of an object reference with image values. 

[0003] Numerous practical uses require objects to be 

automatically rediscovered in images or digital image sequences. 
But It can happen that parts of the object are covered up or 
obscured by obstructions in the field of view. 

[0004] In a known process for rediscovery of objects in 

images, the object image to be located stores a reference and 
that reference is subsequently correlated with the image within 
a limited range. Only part of the reference is used during 
correlation and the image pixels of that reference belong to the 
object. Parts that do not belong to the object are masked by 
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object windowing or object masking. The location of the 
correlation maximum supplies the current position of the object 
in the image, because the image pixels there agree with the 
object pixels, 

[0005] This known process however is problematic in that the 
correlation no longer provides the correct position of the object 
when it happens to be partly covered up, because one correlates 
in the covered area with image pixels that do not belong to the 
object. Due to this, coverup, only part of the object is compared 
to the reference, which leads to an inaccurate recognition of the 
object or of the position of the object. Imprecise recognition 
of the coverage area very quickly yields an erroneous object 
position in the image. 

[0006] It is therefore a purpose of the present invention to 
provide a process for rediscovering objects in images, which will 
facilitate precise recognition and precise determination of the 
object position. 

[0007] Another object of the invention is to provide a system 
for the rediscovery of objects in images, with which the object 
position can be determined precisely. 
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[0008] These and other objects and advantages are achieved by 
the method and apparatus for rediscovery of objects according to 
the invention, which includes a correlation of an object 
reference with image values. If an object is partially obscured 

(covered up) , the image values that are within the coverage area 
in the image, are replaced by gray values of the object or of an 
object reference, before the correlation is performed. In that 
way, the errors connected with the rediscovery of the object are 
significantly reduced . 

[0009] As part of this process, a reference image is 

preferably subjected to interference windowing in order to, 
replace the image values within an interference mask by the gray 
values of the object. Advantageously, the reference image is 
subjected to object windowing in order to get the object 
reference- 

[0010] In the present invention, especially prior to coverup, 
an image of the complete object is stored, and is used to 
determine the position of the object in case of a partial 
coverup- Advantageously, covered parts of the object in the 
image are replaced by parts of a stored reference. 



[0011] The invention-based system for the rediscovery of 

objects in images includes a camera to take a picture, an image 
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data memory to store an object reference, an image data processor 
to replace gray values that lie in the image within a coverup 
area with gray values, of the object reference, and a correlation 
unit that correlates the image - altered by the image data 
processor - with the object reference. The invention-based 
system is able precisely to recognize the coverup area and to 
determine a precise object position in the image. Errors are 
avoided or substantially reduced in that the already stored 
information on the appearance of the object is employed, 

[0012] The invention is guided by a basic cybernetic idea: the 
system stores the image of the complete object and uses this 
information to determine what is behind the coverup. This 
technique is patterned after the human observer, who remembers 
the image of the complete object and who has an idea of what is 
behind the coverup. As a result, one can determine precisely the 
position of the object in case of partially covered objects. 

[0013] Other objects, advantages and novel features of- the 

present invention will become apparent from the following 
detailed description of the invention when considered in 
conjunction with the accompanying drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0014] Figure 1 graphically illustrates the individual steps 

of the invention-based process; and 

[0015] Figure 2 is a flow chart of the invention-based 

process . 

DETAILED DESCRIPTION OF THE DRAWINGS 

'i^ [0016] Figure 1 shows the steps of the invention-based process 
id by way of example, on the basis of target tracking. The target 
5 or object 1 is initially visible, but as time goes on, it 
l:k disappears behind an obstruction 2, which covers it up. At a 

certain point in time, one or more parts la of object 1 can still 
|I be recognized, while the remaining parts of object 1 are behind 

the obstruction 2. If a picture is taken at that point in time, 

it contains object 1 which is partially covered by obstruction 

2 . 

[0017] From a known reference image, which contains the target 
or object 1, an object mask 10 is produced. Object mask 10 is 
superimposed with the picture that was taken and that contains 
target 1 behind obstruction 2. This can be done so long as part 
la of object 1 can still be seen in the image. By 
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superpositioning object mask 10 with the partially obscured 
object and the obstruction, an interference mask 20 is created, 
which is superposed on obstruction 2. That is, interference mask 
20 consists of those parts of the obstruction 2 that cover the 
remaining parts of the object 1 in the image. 

[0018] By means of an image data processor, the image pixels 
inside interference mask 20 are replaced by the gray values of 
object 1 that are taken from the reference image. This results 
in a replacement 21 with image pixels that correspond to just the 
covered parts of object 1. Replacement 21 is then inserted in the 
picture that was taken in the area of interference mask 20 so 
that the image now generated contains the complete object 1, 
without any parts of object 1 missing. The image, thus generated 
with the help of the image data processor, forms a foundation for 
further procedural steps . 

[0019] From the stored reference image (that, as described 

above, contains object 1), a reference or object reference 11 is 
created, with the help of object mask 10. Object reference 11 
is now correlated with image 22 that contains replacement 21. 
The correlation maximum is determined by means of a peak 
detection- The location of the correlation maximum thus supplies 
the precise position of object 1 in the picture. 
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[0020] Figure 2 is a block diagram which shows the process for 
the automatic location or rediscovery of object 1 in the picture 
that was taken. The data of a reference image 201 are subjected 
to interference windowing 202, and those image parts 203 in which 
the object is behind the obstruction are replaced with the gray 
values of the object from the reference image in block 204. The 
reference image is also subjected to object windowing 205 that, 
as a result, supplies the object reference. In a correlation 
unit 205, the object reference is correlated with the image that 
contains the replacement from block 204. Finally, in order to 
determine the correlation maximum and the position of the object, 
a peak detection 207 is performed. (In the process, the reference 
image for example is a previously taken picture that contains 
obj ect 1 . ) 

[0021] In the method and apparatus according to the invention, 
in the search area of the image, the gray values of the 
obstruction are replaced by the gray values of the object; as a 
result, considerably increased precision of recognition and 
position determination of objects is achieved. Moreover, there 
is considerably less intervention in the signals that are subject 
to correlation than during the out-masking of entire image 
portions. Errors during position determination are reduced 
considerably by storing the previously obtained information on 
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the appearance of the object and by using this information to 
find the object behind an obstruction. 

[0022] The foregoing disclosure has been set forth merely to 
illustrate the invention and is not intended to be limiting. 
Since modifications of the disclosed embodiments incorporating 
the spirit and substance of the invention may occur to persons 
skilled in the art, the invention should be construed to include 
everything within the scope of the appended claims and 
equivalents thereof . 
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